Planning with Hidden Parameter Polynomial MDPs

For many applications of Markov Decision Processes (MDPs), the transition function cannot be specified exactly. Bayes-Adaptive MDPs (BAMDPs) extend to consider probabilities governed by latent parameters. To act optimally in BAMDPs, one must maintain a belief distribution over Typically, this is described set sample (particle) MDPs, and associated weights which represent likelihood MDP being true underlying MDP. However, as number dimensions parameter space increases, required sufficiently grows exponentially. Thus, maintaining an accurate form complex spaces computationally intensive, turn affects performance planning for these models. In paper, we propose alternative approach We class BAMDPs where can expressed closed polynomial parameters, outline method closed-form parameters results representation. Furthermore, representation does away with need tune belief. evaluate two domains empirically show that polynomial, closed-form, better plans than sampling-based